We study the sign problem of the fermion determinant at nonzero baryon chemical potential. For this purpose we apply a simple model derived from Quantum Chromodynamics, in the limit of large chemical potential and mass. For SU(2) color, there is no sign problem and the mean-field approximation is similar to data from the lattice. For SU(3) color the sign problem is unavoidable, even in a mean-field approximation. We apply a phase-reweighting method, combined with the mean-field approximation, to estimate thermodynamic quantities. We also investigate the mean-field free energy using a saddle-point approximation.